Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Drop runs-per-test for unit tests #98272

Merged
merged 1 commit into from Jan 22, 2021
Merged

Drop runs-per-test for unit tests #98272

merged 1 commit into from Jan 22, 2021

Conversation

liggitt
Copy link
Member

@liggitt liggitt commented Jan 21, 2021 •
edited

This was added on the assumption that we could use bazel test result caching.

Since we can't this actually doubles the runtime (or resources) of every unit test run.

Without this, flakes can still be seen in the triage board or periodic runs.

Whittling down bazelisms in our test configs ahead of #97921

NONE

/cc @BenTheElder

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jan 21, 2021
@liggitt
Copy link
Member Author

liggitt commented Jan 21, 2021

/kind cleanup
/priority backlog
/triage accepted
/sig testing

@k8s-ci-robot k8s-ci-robot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. priority/backlog Higher priority than priority/awaiting-more-evidence. triage/accepted Indicates an issue or PR is ready to be actively worked on. sig/testing Categorizes an issue or PR as relevant to SIG Testing. approved Indicates a PR has been approved by an approver from all required OWNERS files. and removed do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jan 21, 2021
spiffxp
spiffxp approved these changes Jan 21, 2021
View reviewed changes
Copy link
Member

@spiffxp spiffxp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 21, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@BenTheElder
Copy link
Member

go has a runs per test equivalent (-count) is the test time really too long?

@liggitt
Copy link
Member Author

liggitt commented Jan 21, 2021

go has a runs per test equivalent (-count)

setting that makes you lose the ability to use the test cache :-/

@liggitt
Copy link
Member Author

liggitt commented Jan 21, 2021 •
edited

/retest

hit a (not reproducible locally using stress) flake. oh, how you taunt me.

stress ./options.test -test.run TestServerRunWithSNI
0 runs so far, 0 failures
0 runs so far, 0 failures
0 runs so far, 0 failures
0 runs so far, 0 failures
6 runs so far, 0 failures
10 runs so far, 0 failures
11 runs so far, 0 failures
12 runs so far, 0 failures
16 runs so far, 0 failures
18 runs so far, 0 failures
20 runs so far, 0 failures
23 runs so far, 0 failures
28 runs so far, 0 failures
30 runs so far, 0 failures
31 runs so far, 0 failures
32 runs so far, 0 failures
34 runs so far, 0 failures
39 runs so far, 0 failures
40 runs so far, 0 failures
44 runs so far, 0 failures
44 runs so far, 0 failures
48 runs so far, 0 failures
52 runs so far, 0 failures
56 runs so far, 0 failures
56 runs so far, 0 failures
56 runs so far, 0 failures
61 runs so far, 0 failures
65 runs so far, 0 failures
67 runs so far, 0 failures
68 runs so far, 0 failures
72 runs so far, 0 failures
75 runs so far, 0 failures
78 runs so far, 0 failures
80 runs so far, 0 failures
81 runs so far, 0 failures
85 runs so far, 0 failures
89 runs so far, 0 failures
92 runs so far, 0 failures

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit f8ca5ad into kubernetes:master Jan 22, 2021
@k8s-ci-robot k8s-ci-robot added this to the v1.21 milestone Jan 22, 2021
@liggitt liggitt deleted the unit-test-runs branch January 26, 2021 15:48
jingxu97 pushed a commit to jingxu97/kubernetes that referenced this pull request Mar 2, 2021
@liggitt
Drop runs-per-test for unit tests
abb9816 
Cherry-pick of kubernetes#98272

Related to b/177937814

Change-Id: I8e5e69e22158a3fdf6bb816c83219e4daad1087d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@spiffxp spiffxp spiffxp approved these changes

@BenTheElder BenTheElder Awaiting requested review from BenTheElder

Assignees

@spiffxp spiffxp

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/backlog Higher priority than priority/awaiting-more-evidence. release-note-none Denotes a PR that doesn't merit a release note. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Milestone
v1.21
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@liggitt @k8s-ci-robot @BenTheElder @fejta-bot @spiffxp